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ABSTRACT 

Differences in conditions in the U.S. and one 
developing country, the Bahamas, suggest that evaluation of higher 
education programs are sore welcome in the latter. U.S. evaluation 
programs, *such as Florida's annual review of selected disciplines in 
its nine state institutions of higher education, are fev in number 
and have spread only recently, in response to declining resources and 
increasing demands for accountability. In developing countries, 
however, c there is a critical need for program evaluation, because of 
these countries 9 large number of pro&ieuB, the consequent need for 
saay higher education programs, the lack of models for comparison, 
the sharp limits on resources And funding, and the need to deal with 
political pressures for particular programs* An example of a 
successful evaluation occurred in the Bahamas, where the College of 
the Bahamas assessed the need for and implementation and. outcome of a 
new program to prepare secondary schopl dropouts for employment or 
higher education. The evaluation process included data collection and 
analysis as well as faculty training in evaluation methods. As 
expected, college and government officials were very receptive to the 
program evaluation process. (RB) 
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ABSTRACT ' ' v 

In" this paper the need for a^qewer and more expanded 
conception of program evaluation is described: The theme '* 
is advanced that conditions are favorable for the Lmple- 

4 # 

mentation of, the evaluation process in one of the world's 

/ 

developing countries , the Bahamas , Where, it is pointed * 
oat, resource limitations and intense need combine to maxi- 
mize potential gains as well as the' likelihood of adherence 
to procedures. The paper includes a summary of the 
evaluation strategies the, author has employed in the Bahamas 
and a description of the results with one program.- 
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I. BACKGROUND: Prograri Evaluation-rfew directions/ old 
v * realities 

F.or reasons ' ranging from declining enrollments and 

i • 
declining revenues to legislative demands for accountabi- 

lity, mkny institutions of higher education in the United 

States have begun to implement a process of program review 

arid evaluation. For the piirpose of this paper, program 

evaluation is defined as a sy sterna t ic and ob j ect ive exam- 

inati.on of the key components of an institution's degree 



or non-degree o£fe$ings. It is wide-ranging* enough to- 
include everything from a consideration of the rationale 
for the offering. tTo its programmatic design and follow-up. 

i 

The intensity of its implementation and methodology employed* 
depends upon conditions present in the institution and 
ranges* from trursory looks at crledit hour productivity all 
the way to "sunset" clauses which require the development 
of a convincing rationale for all programs every couple of 
years to insure continued funding. 

The situation in the State of Florida illustrates the 
purposes and the methodology being employed to implement one 
kind of program review. The Florida BoarcLof Regents . ' 
designed a system which would, ^ 9 



"... shift away from* reliance on degree " 4 
productivity criteria as an appropriate measure of / 
program quality and as the -sole means of identifying . 
existing programs which may be subjected tA an in- N 
depth study for possible discontinuation. The n$w 
process of systemwide review is intended t* confirm 
that programs axe meeting standard^ of quality, are 
being managed efficiently, and are providing a 
desired service to their regions and the St ate . * 0 (Hill , 1978) 
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Within the State seve'ral academic disciplines are 

studied intensively each year. Once' these programs are 

identified, each of the nine state institutions of higher 

education develops responses to specific state questions 

dealing with issues such as manpower need for the program, 

professional preparation of program faculty and, among 

many other concerns? Jthe Mature and diversity af instruc- 

S 

tional activities available within, the pro-am • Locally 

prepared written materials are Supplied to a team of out - 

* » 
side consultants jointly selected by the state and the 

institutions. The consultants, themselves equipped w-ith 

* * 

a set' of guidelines provided by the state, tfien visit each 

site. The outcome- of the process 13 a set of' recommenda*- 

tions issued by the consultants dealing with site specific 

concerns and system-wide neeos as wjell. 
. • / 
The intent of the* various approaches to program review 

is clear. At a time when money irfay not be available to ^ , 
continuously fund all possible programs, it appears treason- 
able \to utilize objective information to determine* which 
should be, supported at the same level and which are deserving 
of greater or lesser levels of support. Presumably Reviews • 

could serve the purpose of "saving" or revitalising needed, 

i 

' \ 
but floundering programs, or even lead to' the initiation of 

new programs (using saved monie's) for an area currently not 

f 

being served., * ^ 

Despite the encouraging signs of the emergence of 
program review as described above, the overall status of 
evaluation activities at the college leve 1 in general 



remains at a very low level* As Samuel Ball, Senior Research 1 

Psychologist at ETS has noted, 

H • • . the traditional formal educational insti- 
tutions -* the college? - rarely indulge in. program 
^valuation in any fcSrmal sense of that .term. It seems 
that once a program is installed at the college level 
** it becomes, sufficient unto itself or 'functionally 
0 ' autonomous 1 and it is unlikely that any formal effort 
will occur to gatKer evidence 'Concerning the need for 
program modi-f ication or continuation" . (Ball, 1977) 

The foregoing, suggests that the review and evaluation 

process, long dormant in higher education circles, may, 

be rejuvenated in response to new realities. In order to 

have sustained impact, however, it will first be ^e pessary 

to overaome long standing obstacles and various other 

conditional factors including a lack of familiarity with 

data-based decision making. The remainder of this paper 

is devoted .to consideration of a locale where situational 

factors vary from what is present in thVs country. It 8 

will be pointed out that in the Bahamas factors such as a 

critical need for new programmatic offerings, a lack of 

multiple programs ari^ a severe shortage of resourcfes for > 

program development may serve t,o increase the likelihood 

that the program review process could work within its 

major institution of higher learning. Presented on the 

pages which follow, is a detailed description of the con- 

ditiens which appear likely to enhance the review process 

,in this deveibping country and a- discussion of the exper- 

iences the writer has had in the implementation of the 

process • 



II, PROGRAM REVIEW IN .OTHEK SETTINGS: New Potential 

Is concern for program review warranted in developing 
countries where often ther furfds barely exlgt for the 
establishment of programs ifl the first place? Or is 
program re vi,ew - some thing appropriately done only by those 
institutions, and nations, with spare f unc^^available 
to support it? In short, how likely is it. that the bene- 
fits derived from the process will be sufficiently favorable 
in developing countries to justify involvement with it? 
To begin to address these issues it is worth examining " 
several .'features of the environment present in developing 
countries and relate these to the program review^ process. 

First it should be recognized that institutions of 
higher learning in developing countries have a n£ed for 
pjrogram review data equal to or greater than the need 
of institutions" within "developed" countries £or the fol- 
lowing reasfcnst' 

v 

a. the number of ptroblems in developing countries* 
which may appropriately be' addressed by their higher 
learning institutions is extensive; 

b. the environment within which programs operate 

in developing countries is considerably more flexible than 
it is in developed countries; and ^ 

c. there are fewer higher education based programs 
currently in opetation to serve as models in developing 
•countries chants the case in so-called developed countries* 

. The above suggests that in developing*, countries thdre 
is an extensive need for new programs which must be 
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responsive to changing* conditions . Further, there is 
little -experience to draw upon and, hence, a greater 
likelihood -that , initial attempts may require continuous 
development And, of course, a system fflfr monitoring 

progress and problems. The combination of rapid changes 

•> * * 

and limited experience suggests a Strong nee) for a system 
of program review. 

Secondly, the fact; that funding for program initia- 
tion, and support is so limited in developing countries, 
lea,ds decision bakers 'in their quest to spend scarce 
dollars carefully, f?o'* an increased appreciation of the 
kinci of information the program review process can provide , 
This may stand in contrast to **the situation in many 
sections* of the United States where, as pointed out above, 

f # * • 

evaluation considerations have not been of 'much concern 

* ' , • 

to educators. Only recently, dug to funding constraints 
has this situation even begun to' change. Too often eval- 
uation continues t£ be viewed -as' a periodic nuisance which 
at best will result in programs being left alone until 

• . \ 

the review cycle repeats itself oiVce' again. ' This behavipr 
is not based cm any unwillingness to change, or on any 
organized e'ffort against th.e data collection and review 
process. It is rather a recognition that when critical 
allocation decisions do not have to bft made by higher 
administration^ the review process, will probably end up 
unjased, placed on a shelf somewhere. Receptivity to 
review is not a constant however, and as resource restric- 
tions become evident in American higher education one chn 
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expect significant changes in actiturde toward the 
process ^ f - 

The current situation in developing qoun tries 

stands in sharp contrast to the situation described • 

i +*j ' 

above. In developing countries it is often the case # 
that critical decisions about programs need to be made 
because. the money spent on them cQuld «be used in many 
other .ways. These decisions will -be made whether hard 
objective data j.s or is not available/ In such a setting 
the requirements of prpgram review become no less dif- 
ficult but may be perceived as being far more preferable 
than the alternative of basing' 'decisions on something 
other than the conditions' present in the program in 
question. Findings then tend to be needed in reverse 
proportion to the monies available for program support. 
»One may conclude that institutions with the severest limits* 
on funds (e.g.* those in deve loping countries) stand to 
niaJt/ the greatest use of' data generated in the program 
review process'. * 

Thirdly, because institutions of higher learning in 
developing countries have the least experience with the 
program review process itself and the - super-structure 
within which these institutions operate has little, if 
any ,* experience with program review and all the problems 
which accompany it, there is a need for high placed indi- 

viduals within these institutions to be well trained in 

j 

the implementation of the program review process and 
. strongly committed" to its use. Thus it is recognized 



that even under the best of circumstances decision majcers 
will face political pressures to act in ways not always 
supported by data at their disposal. Training and commit- 
ment will both be necessary to overcome the resource and 
human obstacles^ w^iich tend to accompany program review 
wherever and whenever it is attempted. Further, the 
manager seeking to implement the process cah e xpect that 
he will have to frequently justify his methodology and 
explain his results. 

The poi nts made above indicate a strong need in 
developing countries for the kind of data the program 
review p/ocess can provide; a suggestion that limited 
funding for program support can, if handled properly, 
result in an increased reliance on program review data; 
and a reminder that aeed and livelihood of. use are in- 
sufficient without training ^nd commitment. These factors 
were supported by experiences the writer has had as a 
consultant to the College of the Bahamas in Nassau. The 
College, with Ford Foundation funding and developmental 
assistance provided by the Institute of Open Education 
of Antioch Graduate Center, developed a program to prepare 
non-secbndary school graduates for admission into the.' 
College and/or job advancement. A strong needs statement 
prepared by the College Principal indicated that a signif- 
icant percentage of the Island's population co*uld benefit 
from the Program since, until very recently, v^ry few 
of the nation 1 s citizens had graduated from secondary 
school . 
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Working closely with the College Principal a program 
review model was evolved by the writer with the following 
components : 

a • In-depth training of al 1 faculty involved in 
the Program to acquaiat^t^iem with all phases of the program 
review process and to enable them, by year's end^ to take 
full -responsibility for its continuation. At this point, 
Program faculty are collecting data all consider to be 
important and they are preparing to write the final report. 

b . Introductory seminars on program review were 

presented to all interested' College faculty (25 of total 
% 

faculty of 60 voluntarily^ attended) to acquaint them with 
what was going on in tile Program and to discuss the rele- 
vance of the process to their own interests. 

c. Reviews of progress and problems have been 
presented on a regular basis to College Administration 
along with indications of their 'responsibili ty for the 
institutionalization of the. process. 

d. * A comprehensive data collection, reviey $nd 
analysis |chema is in place which emphasizes the program 
development and enhancement capability of . the ■ program 
review process. The Program staff has pursued the review 
process utilizing a t|iree stage model and, to <}ate, has 

drawn conclusions in each of these~~areas as follows: 
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1. Needs Assessment: The staff sought to learn the extent 

•* 

of the need for their program and likelihood of the need to con- 

* » «• 

tinue over the short term* and- the long term. They discovered an 

alarmingly high dropout rate, which they labeled "epidemic" 'and 

they found no programs in existence, or planned, ^to address the 

problem. • v * v 



As # a result of thisp^ase of the evaluation, staff 

generated statistics oft the incidence of this behavior (for the' 

first time in the Bahamas) and were able to point out the long 

term need for changes in schooling .opportunities* an£ job require 

ments to cope with the problem*. 

§ . * 

2. Program Planning and Implementation: In thia phase of 

their investigation, the staff, studied each component of their 

program to determine how it wa* operating and whet h ^ r x u was con 

tributing to this resolution of the problems identified in their 

needs assessments. They, discovered that too much emphasis was 

placed in some areas of their curriculum (math and science) and 



not enough was paid to others (English and job interview skills) 
They discovered tdo that assumptions in program planning they 
h*d made about the need for full time study wars perhaps not 
correct. That is, having; their program require full time study 
turned out to eliminate many potential studen'ts who could npt ' 
"make that kind, of commitment. 
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They concluded that* their tutorial was extremely 
successful and needed to.be expanded. * Other useful findings 
from this phase were derived from interviewing chairmen of 
varipus departments -of the College of th j Bahamas to determine 
their perceptions of how succeifu^ the special curriculum had 
been . These interviews enabled the program planners to further 
mesh their curriculum with college expectations. 

A final interesting finding having to do with program 
implementation was w that the nature of the student being attracted 

into the program went beyond the high school dropout. Thus, the 

♦ / 
program enrolls high school graduates (as well as dropouts) who 

lack study skills, or confidence, along with some basic skills 

needed to* succeed in college and employment. 

3. Outcome Assessment: In this part of their evaluation 
work, staff looked criti Ally at how successful they had been in 
achieving their original objectives. They were particularly 
interested in learning whether their graduates were able to 
achieve in the regular courses of the College of t^e Bahamas, once 
they were enrolled there, and whether those looking for work were 
more successful than they would have been if they had not enrolled 
in the 'program. 

'By studying the grade transcripts of their graduates 
and a comparable grbup of other weak students, both of which sub- 
sequently enrolled in regular college courses, they were abJLe to 
conclude that their* *ost recent graduates fared better thin the 
otlver students in college courses. They also not that their 
gradutes had a lower dropout^rate than students ^f rom other pro- 
grams. M * ^ 
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Unfortunatel,y , their studies alsQ revealed that much 
still needs to be done to change the attitudes of employers, most 

of whom feel that only college graduates capable of passing special 

I 

examinations (which happen to be completely unrelated to the require- 
ments of 'the job) ar ? e eligible for employment. 



Ill/ SUMMARY 

In this paper the author has suggested that certain 
conditions present in the Bahamas increase the receptivity of 
program and governmental leaders toward the program evaluation 
process. t ' . 

In^the evaluation discussed herein, staff and £n outside 
•valuator, looked critically at a program designed to meet signifi- 
cant needs of their developing country; namely, what to do about the 
young people who drop but of hi"h school unequipped for work or 
further education. TT iing a multi-stfcge needs ■ assessment model, 
conclusions were c_j relative to the pervasiveness of the 

problem, curricula changes needed, and overall impact to date. 



The predicted receptivity to the evaluation process on 
the part of College and governmental officials was, in fact, 

a 

borne out, leading to the conclusion that the prospects for improved 
services to clients and for educational development in general 
'in the Bahamas may be quite favorable. The next group to take 
on are the entrenched employers who continue to deny meaningful 
work to many qualified applicants. 



